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Content-intensive math 
professional development 
improved teachers’ 
KNOWLEDGE and PRACTICE 
but not their students’ 
ACHIEVEMENT. 



* = statistically significant impact at the 0.05 level. 


Focusing on teachers’ 
mathematical knowledge: 
The impact of content-intensive 
professional development 

A 93-hour professional development program 
focused on deepening math content knowledge 
had a positive impact on fourth-grade teachers’ 
knowledge and some aspects of instructional 
practice. It did not, however, have a positive 
impact on student achievement. The impacts 
imply that a teacher with average math 
knowledge (at the 50th percentile) who receives 
the professional development would become 
among the top-third of teachers (with knowledge 
at the 71st percentile). A teacher who is average 
at explaining math concepts in class would 
improve from the 50th to the 73rd percentile. 



The policy context 


Program details 


Recent results from national and international assess- 
ments continue to show a need for improvement in math 
achievement among U.S. students. For example, 60 per- 
cent of grade 4 students scored below the proficient level 
on the 2015 National Assessment of Educational Pro- 
gress.^ 

In an era of increasingly rigorous state standards, teachers 
at all grade levels face heightened expectations to deepen 
their students’ understanding of math concepts. Teachers 
may benefit from professional development (PD) that 
strengthens their own conceptual understanding of math, 
particularly elementary school teachers who are less likely 
to formally study math in college than secondary teachers 
are. To date, there is limited convincing evidence on the 
effectiveness of intensive, content-focused PD, a gap this 
study intended to address. 


^ National Center for Education Statistics. (2015). The nation s 
report card. Washington, DC: Institute of Education Sciences, 
U.S. Department of Education. 
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This study examined the implementation and impact of a 
93-hour PD program that focused on deepening teachers’ 
general math knowledge but also covered some math 
knowledge relevant to teaching. The core of the PD was 
Intel Math, an intensive 80-hour workshop delivered in 
summer 2013 that focused on deepening teachers’ 
knowledge of grades K-8 math.^ Two additional PD 
components. Mathematics Learning Community^ and Video 
Feedback Cycles,"^ totaling 13 hours were delivered during 
the 2013-14 school year to reinforce the math content in 
Intel Math and help teachers apply the content to im- 
prove their instruction. The Mathematics Learning 


^ Intel Foundation. (2009). Intel Math, Version 2.5. Santa Clara, 
CA: Author. 

^ Regional Science Resource Center at the University of Massa- 
chusetts Medical School. (2011). Mathematics Learning Commu- 
nity. Malden, MA: Commonwealth of Massachusetts, Depart- 
ment of Elementary and Secondary Education. 

The Video Feedback Cycle component was based on emerging 
research on individualized feedback for teachers. For example: 
Allen, J. P., Pianta, R. C., Gregory, A, Mikami, A. Y., (Sc Lun, 
J. (2011). An interaction-based approach to enhancing sec- 
ondary school instruction and student achievement. Science, 

333, 1034-1037. 
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Community involved five structured, 2-hour collaborative 
meetings in which teachers reviewed math content and 
analyzed student work on topics covered in Intel Math. 
The Video Feedback Cycles involved three rounds of in- 
dividualized, video-based coaching that provided feedback 
to teachers on the quality and clarity of their mathemati- 
cal explanations. The feedback was based on video analy- 
sis of their lessons using a structured observation rubric 
called the Mathematical Quality of Instruction (MQI).^ 

Study approach 

Grade 4 teachers from 94 schools in six districts and five 
states participated in the study and were randomly as- 
signed within schools to either a treatment group that re- 
ceived the study PD or a control group that did not re- 
ceive the study PD. The final analysis sample included 
165 teachers from 73 schools. The study addressed the 
following questions: 

1. Was the study PD implemented with fidelity? What 
were the features of the PD as implemented? To what 
extent did teachers participate in the PD? 

2. What was the impact on teachers’ content 
knowledge, teachers’ classroom practices, and student 
achievement of offering content-focused PD relative 
to business-as-usual PD? 

To address these questions, we assessed teachers’ math 
content knowledge in summer 2013 (prior to the study 
PD), fall 2013 (after Intel Math), and spring 2014 (after 
the entire study PD was delivered) with an adaptive as- 
sessment covering content emphasized by the study PD. 

We assessed teachers’ instructional practice in fall 2013 
and spring 2014 by video-recording three lessons per 
teacher (instructing their own students), and then rating 
these lessons using the MQI rubric. The MQI was devel- 
oped by education professors at Harvard University and 
the University of Michigan. It was selected because it is a 
structured rubric that is widely used in studies of class- 
room math instruction. The MQI uses a 4'point scale to 
rate three dimensions of instructional practice, which are 


^ Mathematics Instrument Development Group. (2013). Math- 
ematical quality of instruction. Cambridge, MA: Author. 


briefly summarized in Box 1. A score of “not present” in- 
dicates that the practice was not present in the classroom. 
Scores of “low,” “mid,” or “high” indicate that the prac- 
tice was present in the classroom at that particular level. 
For Richness of Mathematics and Student Participation in 
Mathematics, “high” scores correspond to the highest qual- 
ity of instruction. For Errors and Imprecision, “high” scores 
correspond to the lowest quality of instruction because 
they indicate that errors and imprecision were common 
and obscured the math in the lesson. 


Box 1. Dimensions of instructionai practice measured with 
the MQi 


Richness of 
Mathematics 

Emphasizes the conceptual aspects of 
math, including the use and quality of 
mathematical explanations 

Student Participation 
in Mathematics 

Focuses on teachers’ use of student 
mathematical contributions, explana- 
tions, questioning, and reasoning 

Errors and 

Focuses on incorrect, unclear, and im- 

imprecision 

precise use of math 


There were two measures of 4th grade student achieve- 
ment in spring 2014. The first, administered by the study 
team in all study districts, was an adaptive assessment 
aligned with the content covered in the study PD. The 
second was the state math assessment, which provided 
policy-relevant test scores. 

To better understand teachers’ PD experiences and how 
faithfully the PD was implemented, we administered a 
teacher survey in spring 2014 and collected a variety of 
PD implementation data throughout the school year (e.g., 
PD attendance records). 

Findings highiights 

The study found that: 

• The PD was well implemented, with math instruc- 
tional quality evident most of the time and high 
rates of teacher participation. On average, 96 per- 
cent of the expected 80 hours of Intel Math and 100 
percent of the expected 13 hours of the Mathematics 
Learning Community and Video Feedback Cycles 
were delivered. Math instructional quality, measured 
with the MQI, was evident during most of the whole- 
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group discussion of math content.^ On average, 
treatment teachers participated in more than 90 per- 
cent of the delivered hours for each component of 
the PD program (98 percent of Intel Math, 90 per- 
cent of the Mathematics Learning Community, and 
97 percent of the Video Feedback Cycles), Overall, 
treatment teachers reported having participated in 95 
more hours of math-related PD than did control 
teachers during the year of the study. 


significant and positive. An average treatment teacher 
demonstrated Richness of Mathematics at a mid or high 
level during 63 percent of a typical lesson, compared 
with 46 percent for an average control teacher (Figure 
2),^ This 17 percent difference corresponds to an im- 
provement of 23 percentile points. The impact on 
Student Participation in Mathematics and Errors and Im- 
precision in the spring were in the expected direction 
but not statistically significant. 


• The PD had a positive impact on teacher knowledge* 

Treatment teachers’ average knowledge score was 7 
points higher than control teachers’ average score in 
the fall and 6 points higher in the spring (Figure 1), 
These differences imply that a typical control teacher 
would have improved from the 50th percentile to the 
74th percentile in the fall and to the 71st percentile 
in the spring, had the teacher received the study PD, 


Figure 1. Teacher knowledge scores 
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Source: Fall 2013 and spring 2014 teacher knowledge tests (A/ = 
73 schools; 79 treatment teachers and 86 control teachers). The 
teacher knowledge test was based on items created by the 
Northwest Evaluation Association (NWEA). The scores are re- 
ported on the scale used by the test developer, which takes into 
account the difficulty of individual test items. 

* Impact is statistically significant at the 0.05 level, two-tailed test. 

• The PD had a positive impact on teachers^ Richness 
of Mathematics in the classroom* The study meas- 
ured instructional practice by scoring video-recorded 
lessons of teachers on the three MQI dimensions of 
instructional practice (see Box 1). The PD’s effect on 
Richness of Mathematics in the spring was statistically 


^ “Evident” means that Richness of Mathematics and Student 
Participation in Mathematics were present at a low, mid, or high 
level, while Errors and Imprecision were not present. 


Figure 2. Percentage of an average teacher’s lesson 
demonstrating three dimensions of Mathematical 
Quality of Instruction 
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Source: Spring 2014 MQI scores (N = 73 schools; 79 teachers 
and 158 lessons for the treatment group and 86 teachers and 172 
lessons for the control group). 

* Impact is statistically significant at the 0.05 level, two-tailed test. 


• Despite the PD^s positive impact on some teacher 
outcomes, the PD did not have a positive impact on 
student achievement* On average, treatment teach- 
ers’ students scored 2 percentile points lower than 
control teachers’ students on both spring 2014 stu- 
dent achievement measures (see Figure 3). The differ- 
ence between treatment and control group students 
was statistically significant for the state math assess- 
ment but not the study-administered assessment.^ 


^ “Demonstrated” means that teachers were rated at a mid or 
high level on one or more elements of the Richness of Mathe- 
matics dimension (e.g., linking representations, mathematical 
sense-making, multiple procedures or solution methods). 

® However, the impact on state assessments was sensitive to 
sample definition and the inclusion of covariates. It was not 
statistically significant in any of our sensitivity analyses. 
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Figure 3. Student math scores 

■ Treatment ■ Control 


250 


240 


230 


220 - 


210 


S 200 

T3 

^ 190 


180 


215 


217 


100 


o 

o 


80 


in 

^ 60 

*s 

5) 40 


c 

o 

■D 

3 


20 - 


</) 


48 


50 


Source: Spring 2014 NWEA test (A/ = 73 schools; 79 teachers 
and 806 students in the treatment group; 86 teachers and 891 
students in the control group). District administrative records of 
Spring 2014 state test (A/ = 73 schools; 79 teachers and 1,760 
students in the treatment group; 86 teachers and 1,917 students 
in the control group). The NWEA score is reported on the scale 
used by the test developer, which takes into account the difficulty 
of individual test items. The state score is reported as Normal 
Curve Equivalents, which run from 0 to 100 and are similar to 
percentile ranks but on an equal-interval scale. 

* Impact is statistically significant at the 0.05 level, two-tailed test. 

• Teacher knowledge and instructional practices 
were generally not correlated with student 
achievement* The study PD assumed that teachers’ 
content knowledge is related to instructional prac- 
tice, which in turn is related to student achievement. 
Contrary to these assumptions, both knowledge and 
instructional practice in the study were generally not 
statistically significantly associated with student 
achievement (estimates of association between 0.00 
and -0.05). The only teacher measure statistically 
significantly associated with student achievement 
was the Errors and Imprecision dimension of instruc- 
tional practice (estimates of association between 
-0.20 and -0.21). A teacher who had higher Errors 
and Imprecision was predicted to have lower-scoring 
students on both the state and study-administered 
math assessments. 


Concluding thoughts 

Together these results show that the study PD improved 
teachers’ knowledge and some aspects of classroom practice 
but did not improve student achievement. This may be 
partially explained by the finding that math content 
knowledge and instructional practice, as measured in this 
study, were generally not correlated with student math 
achievement. The one exception to this pattern was Errors 
and Imprecision, on which the study PD did not have a sta- 
tistically significantly impact. Thus, future research might 
focus on identifying PD that will improve this aspect of 
practice. Future research might also seek to identify other 
aspects of knowledge and practice to target with PD that 
are more strongly related to improved student achieve- 
ment. 


lES develops these study snapshots to offer short, accessible summaries of complex technical evaluation reports. For the full 
report with technical details, see 

Garet, M. S., Happen, J. B., Walters, K., Parkinson, J., Smith, T. M., Song, M., Garrett, R., Yang, R., & Borman, G. D. (2016). 
Focusing on mathematical knowledge: The impact of content-intensive teacher professional development {HCEE 2016-4010). 
Washington, D.C.: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. 
Department of Education. 
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